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(57) Abstract 

An interactive 
audio-visual (A/V) 
transceiver is advantageously 
coupled to a television 
and/or telephone (T/T) 
cable, a TV, a video recorder 
(VCR), and other A/V 
devices. The A/V transceiver 
switches data between a 
program/service provider 
the connected A/V 
devices. In one embodiment, 
transceiver includes 
three primary modules, a 
main module including a 
CPU, a system bus, system 
memory, an infra-red (TR) 
control unit, an audio-visual 
bus, an A/V decoder, an 
A/V processor, and an A/V 
der, an A/V connect 
module including a number 
tuner/demodulators and 
switch, and an optional 

CD ROM module. The A/V transceiver hardware is complemented with an operating system and software program which supports the 
functions provided in the A/V user interface. Additionally, a remote control device is provided to communicate with the A/V transceiver to 
interactively manage selection of program and service sources, selection program and service offerings from any selected source, viewing 
of selected program offerings, and interaction with selected service offerings. The remote control device is advantageously provided with 
a basic A/V control button group, an interactive control button group, an auxiliary control button group and a numeric key pad to facilitate 
control of the transceiver. The interactive control button group includes an info button, a list button, a categories button, a pix button, a 
mark button, a jump button, and a pointing device consisting of up, down, left, and right arrow buttons, and a center select button. 
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METHODS AND APPARATUS FOR MANAGING VIEWING OF 
AND INTERACTION WITH AUDIO- VISUAL PROGRAM AND 
SERVICE OFFERINGS 
RELATED APPLICATIONS 



The present application is related to the following 
contemporaneously filed applications: 

(a) Serial No. , entitled "An Audio- Visual 

System For Selectively Viewing And Interacting With Programs And 
Services From A Number of Program /Service Sources"; 

(b) Serial No. , entitled "Method And Apparatus 

For Controlling An Audio- Visual System For Selectively Viewing 
And Interacting With Programs And Services From A Number of 
Program/Service Sources"; 

(c) Serial No. , entitled "Methods And 

Apparatus For Managing Selection Of Audio-Visual Program And 
Service Offerings"; 
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BACKGROUND OF THE INVENTION 

1. Field of fre Invention; 

The present invention relates to the field of audio-visual 
systems. More specifically, the present invention relates to a system 
for selectively viewing and interacting with programs and services 
from a number of program /service sources, a control device for 
controlling the system, and the methods and apparatus incorporated 
in the system for managing selection, viewing, and interacting with 
the program/service offerings. 

2. Art Background: 

Over the past 40 years, television and motion pictures 
have become an important aspect of everyday life for people in the 
industrialized world. The development of advanced technology in 
the areas of digital and high definition television (HDTV), video 
recording systems, laser disks and compact disc (CD) entertainment 
systems, coupled with satellite, cable television and telephone 
services, have provided opportunities for viewers to store, retrieve 
and selectively display a variety of television and audio-visual or 
interactive programming on home entertainment systems. Over the 
past decade, improvements in personal computing systems have 
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provided a variety of powerful miniaturized personal computers 
which permit the storage of data and control of home appliances, 
such as entertainment systems, through the use of microprocessors. 
Additionally, a variety of graphic user interfaces have been developed 
to ease human interaction with these new personal computer 
systems. 

The combination of computer technology with 
television (TV) and audio-visual (A/V) systems, has fostered the 
development of multi-media interactive entertainment systems. The 
combination of computer hardware and software with audio-visual 
systems has brought new forms of art and entertainment into being. 
CD read only memory (ROM) systems coupled to personal computers 
permit interactive video computer simulations, unique self -paced 
learning environments and interactive "movies", not possible in 
earlier television systems (See, "The Revolution Starts Here", 
Newsweek, page 42 (January 18, 1993)). It is known that consumers 
desire interactive television and multi-media applications for home 
use. One of the biggest challenges for an interactive television service 
is the design of an interface that is easy and fun to use by average 
consumers. 

In the computer industry, a variety of graphic user 
interfaces have been developed to facilitate human interaction with 
computer systems. Many display systems utilize metaphors in the 
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design of the interface as a way of maximizing human familiarity, 
and conveying information between the user and the computer. It is 
well known that designing around a familiar metaphor helps reduce 
human learning time (See for example, Patrick Chan, "Learning 
Considerations In User Interface Design: The Room Model", Report 
CS-84-16, University of Waterloo Computer Science Department, 
Ontario, Canada, July, 1984, and the references cited therein). 
Research in interface design using metaphors in man-machine 
interfaces may be applied to multi-media systems, and in particular, 
to interactive television systems. 

The marriage of video and television technology with 
computer interface technology provides consumers with maximum 
flexibility in storing, retrieving and viewing television and other 
audio-visual programming. As will be described, the present 
invention provides methods and apparatus for presenting an 
improved audio-visual user interface, which includes various user- 
selectable features for viewing and controlling a television, video tape 
recorder (VCR) and other audio-visual devices. As will be disclosed, 
the present invention's user interface provides a user-friendly 
mechanism for consumers to view, record, and play back TV and 
A/V programs, as well as control other A/V home entertainment 
devices using a remote control device. Furthermore, information 
such as TV program listings and additional information related to 
programs as well as selecting and controlling categories of interactive 
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programs and services may be provided through the user interface of 
the present invention. 
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SUMMARY QF THE INVENTION 

The present invention provides methods and apparatus 
for presenting an improved audio-visual user interface for selecting 
and displaying cable television or other audio-visual programs, as 
well as controlling various audio-visual devices and interactive 
services. The present invention assumes a service provider provides 
cable television and/or telephone (T/T) service to users via a T/T 
cable, including a digital channel of program /service listings, at least 
one digital back channel (from the user's home to a central file 
server), a number of analog TV channels, a number of digital pay-per- 
view channels, and other interactive services, transmitted from 
remote storage devices such as digital file servers. Under the present 
invention an interactive audio-visual (A/V) transceiver is 
advantageously coupled to a user's television, video tape recorder, 
and the T/T cable. In one embodiment, the transceiver includes three 
primary modules. A main module includes a central processing unit 
(CPU) coupled over a digital system bus to system memory and, 
preferably, an infra-red (IR) or similar wireless control unit. The 
main module is further coupled over an audio- visual bus to an A/V 
decoder, an A/V processor, an A/V encoder, a TV, an A/V connect 
module connected to the T/T cable, and to one or more VCRs and 
other A/V devices. The main module may also be coupled to an 
optional CD ROM module, to permit the playback of multi-media CD 
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ROM titles, audio CDs, and the like. Other optional modules or 
external devices (such as additional VCRs, CDs, hard disks, telephone, 
fax and answering machines) may be connected to this transceiver 
and controlled through the use of the audio-visual user interface of 
the present invention. The A/V transceiver hardware is 
complemented with an operating system software program which 
supports the functions provided in the present invention's audio- 
visual user interface. 

In operation, the cable or telephone service provider 
transmits an interleaved data stream preferably including a television 
program listing, together with program titles, program times, 
categories, channel numbers and the like, from a central file server on 
the digital channel of program listings to the transceiver. The data is 
interleaved such that the current day's data is followed by other 
weekly data. 

A remote control device is preferably provided for 
communicating with the transceiver, and includes a number of user 
selectable interactive functions such as: an info button, a list button, a 
categories button, a pix button, a mark button, a jump button, a select 
button, and a pointing device consisting of a set of cross-hair arrow 
buttons (up, down, left, right) or other pointing devices capable of 
providing the same functions (such as a mouse, thumb stick or touch 
pad). While viewing the TV, a user may obtain additional 
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information on a current program by depressing the info button, and 
obtaining more detailed information using the pointing device. By 
depressing the list button on the remote control device, the 
transceiver displays a program listing of the current programs 
available for viewing. Through the use of the pointing device, 
viewers can scroll up and down the program listing or view a 
highlighted program in full screen by pressing the select button. By 
pushing the right or left arrow buttons on the pointing device, 
program listings may also be viewed for different hours, days and up 
to several weeks in advance through the present invention's use of 
data interleaving. The depression of the categories button on the 
remote control device results in the display of a categories menu bar 
on the TV screen, which includes categories such as "all", "sports", 
"news" and "favorites". The selection of one of these categories 
results in the display of only programs within that category. 
Moreover, through the depressions of both the list button and the 
categories button, a program listing is displayed of the currently 
available programs within the selected category. Through the use of 
the arrow buttons on the pointing device, category listings for 
different times and dates up to several weeks in advance may also be 
viewed. The selection of the favorites category results in the display 
of currently available favorite programs, including programs that are 
frequently watched by a particular TV household, marked programs, 
or programs that match preselected user preferences. 
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The depression of the pix button results in the display of 
a "visual menu" of multiple picture-in-picture (PIP) windows along 
the perimeter of the TV screen. Each of the PIP windows displays one 
of the programs currently being received by the A/V transceiver in 
the last category selected by the viewer. The PIP windows are 
displayed at a variable frame rate (N) (depending on the number of 
windows), and are captured by one or more tuners in the transceiver. 
A center PIP window displays a larger video image (along with audio) 
of the currently highlighted PEP window. The other PEP windows can 
be similarly viewed by using the pointing device on the remote 
control. The depression of the mark button allows users to 
"bookmark" a particular program for later viewing. The depression 
of the jump button allows a user to jump between previously marked 
programs. 

Additional features, functions, and interface screens are 
available to the viewer using the teachings of the present invention, 
including a menu button for controlling other devices connected to 
the A/V transceiver (such as CD, VCR, etc.), a record button for 
making copies of programs onto connected A/V recording devices 
(such as VCRs, hard disks, etc.), a plurality of control buttons for 
playing back, stopping, rewinding, or fast-forwarding audio-visual 
programs on connected A/V playback devices, a numeric keypad for 
dialing channel numbers, confirming financial transactions with 
personal identification numbers, an optional microphone with talk 
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button for supplementing the present user interface with direct voice 
commands through the use of voice recognition technology, a pay-per 
view interface, and a home shopping interface, as well as other 
features described herein. 



WO 95/01057 



PCT/US94/06941 



11 

BRIEF DESCRIPTION QF THE DRAWINGS 



FIGURE 1 is a functional block diagram of the audio- 
visual (A/V) system of the present invention. 

FIGURE 2 is a functional block diagram illustrating the 
major components of the audio-visual transceiver of Figure 1 in 
further detail. 

FIGURE 3a diagramatically illustrates the electronic 
spectrum utilized by the present invention. 

FIGURE 3b conceptually illustrates the present 
invention's use of an interleaving data stream to download 
program/ service listings, and other information, to the present 
invention's A/V transceiver. 

FIGURE 4a illustrates one embodiment of the remote 
control device of the present invention with the sliding panel in a 
closed position. 

FIGURE 4b illustrates the first embodiment of the 
remote control device of the present invention shown in Figure 4a 
with the sliding panel in an open position. 
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FIGURES 5a and 5b illustrate a second embodiment of 
the remote control device of the present invention with no sliding 
panel and additional features. 

FIGURE 6 illustrates the present invention's full screen 
viewing function as viewed by a user with a graphic overlay panel 
which includes the current channel number as well as the channel 
logo and identifier. 

FIGURE 7 illustrates the present invention's info 
function showing the first page of the basic information track ("info- 
track"). 

FIGURE 8 illustrates the present invention's info 
function with a second page of basic information, obtained by using 
the right arrow button on the remote control device. 

FIGURE 9 is an additional illustration of the info 
function of the present invention in which the user has requested a 
third page of information using the remote control device. 



FIGURE 10 illustrates the present invention's use of the 
info function providing the user with additional tracks of 
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information (such as the "Tid Bits" information track), by using the 
down arrow button of the remote control device. 

FIGURE 11 is a further illustration of additional 
information tracks ("info-tracks"), such as this "Info Mart", accessible 
to the user of the present invention using the down arrow button of 
the remote control device. 

FIGURE 12 illustrates the present invention's use of the 
list function to display current program/ service listings and to 
highlight a particular program. Viewers may scroll up or down that 
list by using the up or down arrow buttons on the remote control 
device. 

FIGURE 13 illustrates the list function of the present 
invention in which the NTV programming during the hour of 8:00 
to 9:00 p.m. includes two programs, one starting at 8:00 p.m., the other 
starting after the previous program has ended. 

FIGURE 14 illustrates the list function of the present 
invention which allows the user to highlight the second program, 
which starts at 8:10 p.m., using the right arrow button of the remote 
control device. 
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FIGURE 15 is an additional feature of the list function of 
the present invention where the user marks the highlighted program 
with the mark button of the remote control device. 



FIGURE 16 illustrates the selection of an alternate hourly 
time slot beginning at 9 p.m. using the right arrow button of the 
remote control device. 



FIGURE 17 illustrates the selection of an alternate day for 
the program /service listing by depressing the right arrow button of 
the remote control device. 

FIGURE 18 illustrates displaying of a record panel 
confirming the title and length of the program to be recorded, along 
with a confirmation of which VCR and tape to record on, by 
depressing a select button in the remote control device. 

FIGURE 19 illustrates the present invention's use of the 
select button on the remote control device while the program/service 
listing is displayed to instruct the selected VCR or A/V recording 
device to make a copy of the highlighted program. 

FIGURE 20 illustrates the present invention's 
"reminder" feature, which automatically displays a live picture-in- 
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picture (PIP) window of a previously marked program when the 
program is received by the A/V transceiver. 

FIGURE 21 illustrates the present invention's jump 
function, wherein pressing a jump button one the remote control 
device permits a user to jump directly to a full screen view of a 
previously marked program, and wherein pressing the jump button 
again results in the display of another marked program. 

FIGURE 22 illustrates the categories function of the 
present invention which is displayed by pressing the categories button 
on the remote control device. 

FIGURE 23 further illustrates the categories function 
wherein the "favorites" category is highlighted by pressing the right 
arrow button on the remote control device. 

FIGURE 24 further illustrates the categories function 
wherein the "sports" category is highlighted by depressing the right 
arrow button on the remote control device. 

FIGURE 25 illustrates a representative television image 
in the case where the "sports" category has been selected by depressing 
the select button on the remote control device. 
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FIGURE 26 is a further illustration of a representative 
television image in which the "sports" category has been selected, and 
an alternate sports channel has been further selected by a user 
through the use of the up arrow button on the remote control device. 

FIGURE 27 illustrates a sports program listing in which 
the list button has been depressed after the "sports" category has been 
selected. 

FIGURE 28 illustrates the categories function in 
conjunction with the list function. 

FIGURE 29 further illustrates the categories and list 
functions when the down arrow button is depressed on the remote 
control device, breaking down the "sports" category into sub- 
categories such as baseball or football. 

FIGURE 30 is a further illustration of the categories and 
list functions in which the category "favorites" is highlighted and can 
now be selected by depressing the select button or broken down into 
sub-categories by depressing the down arrow button on the remote 
control device. 
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FIGURE 31 further illustrates the categories and list 
functions, where the "favorites" category has been broken down into 
a sub-category of "frequent" programs which are currently listed. 

FIGURE 32 is a further illustration of the categories and 
list functions, where another "favorites" sub-category, "marked 
programs", is highlighted displaying a list of previously marked 
programs. 

FIGURE 33 conceptually illustrates the pix function of 
the present invention wherein multiple PIP windows are displayed 
by pressing the pix burton on the remote control device. 

FIGURE 34 illustrates the pix function of the present 
invention in which a new PIP window has been highlighted and is 
displayed in a larger format in the center of the display screen. 

FIGURE 35 is a further illustration of the present 
invention's pix function in which, through the use of the remote 
control device, a PIP window showing a pay-per-view program has 
been highlighted and is displayed in the center portion of the screen. 

FIGURE 36 illustrates the menu function which permits 
users to press a menu button to switch the television display to other 
audio-visual devices coupled to the transceiver, such as VCR, CD, on- 
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line services, telephone, etc. For example, once VCR is selected, with 
the right arrow button, the TV displays an image from the videotape 
currently in the VCR and the user can press the play button on the 
remote to playback a previously recorded program on the video tape. 

FIGURE 37 illustrates one embodiment of the present 
invention for selecting a pay-per-view channel offering entitled 
"Jeff's World" through the use of the select button of the remote 
control device. 

FIGURE 38 illustrates a preview feature of pay-per-view 
under the above illustrated embodiment. 

FIGURE 39 illustrates a credits feature of pay-per-view 
under the above illustrated embodiment. 

FIGURE 40 illustrates an info feature of pay-per-view 
under the above illustrated embodiment. 

FIGURE 41 illustrates a confirmation feature for 
ordering a pay-per-view offering through the use of the numeric 
keypad on the remote control device under the above illustrated 
embodiment. 
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FIGURE 42 illustrates the reminder feature advising the 
user that the selected offering of pay-per-view will begin in three 
minutes under the above illustrated embodiment. 

FIGURE 43 illustrates one embodiment of the present 
invention for selecting and interacting with a home shopping service 
(TV Shop) offering. 

FIGURE 44 illustrates a sample commercial shown to the 
user if the home shopping service is selected through the use of the 
select button on the remote control device under the above illustrated 
embodiment. 

FIGURE 45 illustrates a menu permitting the user to 
select various shops available on the home shopping service using 
the select button on the remote control device under the above 
illustrated embodiment. 

FIGURE 46 is a further illustration of the home 
shopping service in which a cookware product is highlighted under 
the above illustrated embodiment. 



FIGURE 47 is a further illustration of the home 
shopping service in which a mini-espresso machine is highlighted, 
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and can be selected for ordering with the select button on the remote 
control device under the above illustrated embodiment. 

FIGURE 48 is an information screen provided to the user 
after selecting a mini-espresso machine for ordering under the above 
illustrated embodiment. 

FIGURE 49 is an order screen in which the user, using 
the numeric keypad on the remote control device, enters a personal 
identification number to order the mini-espresso machine under the 
above illustrated embodiment. 



FIGURE 50 is a sample order confirmation provided to 
the user subsequent to ordering a product in the home shopping 
service under the above illustrated embodiment. 
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NOTATION AND NOMENCLATURE 



The detailed descriptions which follow are presented 
largely in terms of interface display images, algorithms, and symbolic 
representations of operations of data bits within a computer memory. 
These algorithmic descriptions and representations are the means 
used by those skilled in the data processing arts to most effectively 
convey the substance of their work to others skilled in the art. 

An algorithm is here, and generally, conceived to be a 
self consistent sequence of steps leading to a desired result. These 
steps are those requiring physical manipulations of physical 
quantities. Usually, though not necessarily, these quantities take the 
form of electrical or magnetic signals capable of being stored, 
transferred, combined, compared, displayed and otherwise 
manipulated. It proves convenient at times, principally for reasons of 
common usage, to refer to these signals as bits, values, elements, 
symbols, characters, images, terms, numbers, or the like. It should be 
borne in mind, however, that all of these and similar terms are to be 
associated with the appropriate physical quantities and are merely 
convenient labels applied to these quantities. 

In the present case, the operations are machine 
operations performed in conjunction with a human operator. Useful 
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machines for performing the operations of the present invention 
include general purpose digital computers, digitally controlled 
displays or other similar devices. In all cases, there should be borne 
in mind the distinction between the method operations of operating a 
computer and /or display system, and the method of computation 
itself. The present invention relates to method steps for operating a 
computer and interactive display system, and processing electrical or 
other physical signals to generate other desired physical signals. 

The present invention also relates to apparatus for 
performing these operations. This apparatus may be specially 
constructed for the required purposes or it may comprise a general 
purpose computer selectively activated or reconfigured by a computer 
program stored in the computer. The method steps presented herein 
are not inherently related to any particular computer or other 
apparatus. In particular, various general purpose machines may be 
used with programs in accordance with the teachings herein, or it 
may prove more convenient to construct more specialized apparatus 
to perform the required method steps. The required structure for a 
variety of these machines will appear from the description given 
below. Machines which may perform the functions of the present 
invention include those manufactured by the Assignee, Apple 
Computer, Inc., as well as other manufacturers of computer and 
computer controlled muti-media systems. 
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DETAILED DESCRIPTION OF THE INVENTION 



The following detailed description will be divided into 
several sections. The first of these will describe a general system 
arrangement for receiving and generating the audio-visual user 
interface of the present invention. Subsequent sections will deal with 
the functional aspects such as the present invention's interactive 
functions, which includes the info, list, record, mark, jump, category, 
pix and menu functions, as well as the overall structure and 
operation of the present invention's user interface. 

In addition, in the following description, numerous 
specific details are set forth such as functional blocks representing data 
processing devices, and metaphors such as screen, menu and other 
configurations to assist the user in navigating through the user 
interface, etc., to provide a thorough understanding of the present 
invention. However, it will be apparent to one skilled in the art that 
the present invention may be practiced without these specific details. 
In other instances, well known circuits and structures are not 
described in detail so as not to obscure the present invention 
unnecessarily. For the purpose of this application, the word "audio- 
visual" and its abbreviation, "A/V", will be used interchangeably and 
will have the same meaning. 
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G-eneral System Configuration 

The multi-media interactive television system of the 
present invention is illustrated in schematic form in Figure 1. As 
shown in Figure 1, a cable or telephone service provider 50 provides 
cable television or telephone (T/T) services over a T/T cable 52 to a 
plurality of users coupled to the cable or telephone system, as is 
known. Although in this Specification reference is made to a cable 
television or telephone system, it will be appreciated by one skilled in 
the art that the present invention may be used in conjunction with a 
variety of other electronic transmission systems including satellite 
service systems, microwave systems, fiber optic, and radio frequency 
(RF) systems. 

As illustrated in Figure 1, the T/T cable 52 is coupled to 
an audio-visual transceiver 54 which comprises a number of 
separately identifiable modules. The transceiver 54 is intended to be 
located in proximity to and coupled to a VCR 56, a television (TV) 58, 
as well as one or more optional audio-visual devices 57 such as 
additional VCRs, laser disc players, camcorders, stereos, various 
storage devices, telephones, faxes and answering machines, as shown. 
Moreover, it is contemplated that transceiver 54 may be directly 
incorporated into the VCR 56 , the TV 58, or one the other A/V 
devices 57. As will be described below, a remote control device 60 
communicates with the transceiver 54 preferably through a wireless 
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transmission signal (for example, an infrared (IR) signal), or other 
mechanisms known in the art. Additionally, as illustrated in Figure 
1, the transceiver 54 is further capable of communicating with the 
television 58, the VCR 56 and the other A/V devices 57 through 
infrared or other means. 

Referring now to Figures 1 and 2, the transceiver 54 
comprises three primary modules: a main module 62, an A/V connect 
module 66 and an optional compact disc read only memory (CD ROM) 
module 70. The main module 62 includes a central processing unit 
(CPU) 63 coupled over a system bus 64 to a system memory 65 and an 
infra-red (IR) control unit 82, which sends and receives wireless 
control signals to and from the remote control device 60. The CPU 63 
is further coupled through the system bus 64 to a memory and bus 
controller 80, which is itself coupled through an A/V decoder 74 and 
an A/V encoder 78 to the A/V connect module 66, as well as to an 
optional CD ROM module 70. The CPU 63 is also coupled through the 
system bus 64, the memory and bus controller 80 and an A/V bus 73 to 
an A/V processor 77 and an optional A/V memory 75. 

The A/V connect module 66 switches and receives 
analog audio-visual signals and digital data from a plurality of audio- 
visual sources including the T/T cable 52, the video cassette recorder 
(VCR) 56, or the other A/V devices 57, and couples those signals and 
data to the main module 62 through the video decoder 74. The A/V 
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connect module further switches audio-visual signals and data 
received from the main module 62 through the video encoder 78, and 
sends them back out to the T/T cable 52, the VCR 56, the other A/V 
devices 57 and/or the TV 58. The A/V connect module 66 includes a 
switcher 67 and one or more programmable tuners /demodulators 69, 
wherein one tuner/ demodulator reads and displays a current 
program from one of the channels received over the T/T cable 52, and 
additional tuners /demodulators (or the same tuner /demodulator, 
used in alternation) are used to read and display data from the side- 
band channels in picture-in-picture (pip) windows. In addition, the 
A/V connect module 66 may include descrambling circuitry (not 
shown) to descramble premium programs received over the T/T cable 
52. It is further contemplated that the A/V connect module 66 
provides a graphic overlay function that superimposes an A/V signal 
from the video encoder 78 against another A/V signal from the T/T 
cable 52, the VCR 56, or the other A/V devices 57 allowing both 
signals to be simultaneously displayed on the TV 58, the VCR 56 or 
the other A/V devices 57. Finally, the A/V connect module 66 can be 
used to transmit data such as order information to the cable (T/T) 
service provider 50 over a back channel 102 described below. 

The A/V decoder 74 is used to decode data encoded in 
the vertical blanking interval or special side-band cable channels into 
digital data and couple them to the CPU 63 through the memory and 
bus controller 80 and the system bus 64. Furthermore, the A/V 
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decoder 74 is used to convert analog audio-visual signals from the 
A/V connect module 66 into digital A/V data and couple them to the 
A/V processor 77 through the memory and bus controller 80 and the 
A/V bus 73. Moreover, the A/V decoder 74 may be used to resize 
audio-visual signals from the A/V connect module 66 in order to 
create picture-in-picture windows and the like. It is also contemplated 
that the A/V decoder 74 may be used to decompress certain analog or 
digital signals (such as MPEG motion video and the like) and couple 
them to the A/V processor 77 through the memory and bus controller 
80 and the A/V bus 73. 

The memory and bus controller 80 is used to route data 
and control signals between the system bus 64, the A/V bus 73, the 
A/V decoder 74, the A/V encoder 78 and the optional CD ROM 
module 70. The A/V bus 73 is a high-speed digital bus used to free up 
the system bus 64 during the transmission of large amount of audio- 
visual data between the A/V processor 77, the optional A/V memory 
75 and, through the memory and bus controller 80, the A/V decoder 
74, the A/V encoder 78, and the optional CD ROM module 70. The 
A/V processor 77 coupled to the A/V bus 73 is used to manipulate, 
process, render, mix and otherwise re-arrange digital data into 
coherent audio-visual displays. An optional A/V memory module 75 
also coupled to the A/V bus 73 can be used to store A/V data before or 
after processing by the A/V processor 77 (using components such as 
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video random access memory (VRAM) to hold, for example, a frame 
buffer equivalent to one 640 x 480 x 8-bit color image in memory). 

The A/V encoder 78, coupled to the A/V bus 73 through 
the memory and bus controller 80, is used to convert digital A/V data 
from the A/V processor 77 or the optional A/V memory 75 into 
analog audio-visual signals which are routed to the TV 58, the VCR 56 
or the other A/V devices through the A/V connect module 66. It is 
also contemplated that the A/V encoder 78 may be used to encode 
other data such as order information sent by the CPU 63 through the 
memory and bus controller 80 and the system bus 64 and transmit 
them to the service provider 50 through the A/V connect module 66 
and a back channel 102 described below. 

The CPU 63 is also coupled over the system bus 64 to a 
system memory 65 including both volatile and non-volatile memory 
components. The non-volatile part of system memory 65 includes 
read-only memory (such as ROM), which is used to store an operating 
system and playback software, fonts, sounds and the like used in the 
present invention. The non- volatile part of system memory 65 also 
includes rewritable memory (such as SRAM), which is used for 
persistent storage of mark or record indicators, listings of programs 
viewed or taped, and other user preferences. Additionally, the 
volatile part of system memory 65 includes sufficient random access 
memory (such as RAM or DRAM) for the temporary storage of data 
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received over the T/T cable 52 or from the other devices 56 - 58 
connected to the transceiver 54. 

The transceiver 54 can also include an optional CD ROM 
module 70 coupled through the memory and bus controller 80 to the 
system bus 64 and the CPU 63. This optional CD ROM module 70 
constitutes a general purpose storage device to permit playback of 
optical compact discs (CDs), including multi-media CD ROM titles, 
audio CDs, photo CDs, and motion picture CDs on the TV 58. It is 
contemplated that those transceivers which do not include an 
internal CD ROM module could instead include a CD ROM interface 
allowing users to connect an external CD ROM drive to the 
transceiver 54 at a later date. It is also contemplated that this CD ROM 
module 70 may in the future use different electronic, magnetic, optical 
or storage technologies other than the current CD ROM embodiment. 

Additional modules may be added to the transceiver 54 
including, for example, such options as a small computer system 
interface (SCSI) for accessing digital storage devices such as hard disks, 
a modem for exchanging digital data over telephone lines, or a serial 
port for controlling other devices over a wired connection. Another 
module presently being contemplated is a remote beeper button for 
locating the remote control device 60. Many consumers often 
misplace their remote control devices, and it is contemplated that the 
present invention may provide a function wherein a speaker on 
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remote control 60 would beep while the beeper button is pressed until 
the user locates the remote control. 

Referring now to Figure 3a, there is shown one possible 
electronic spectrum of signals provided by the cable T/T service 
provider 50 over the T/T cable 52 to the transceiver 54. As illustrated 
in Figure 3a, the spectrum includes at least one digital program listing 
channel 100, a plurality of back channels 102, and a plurality of 
standard analog TV channels 103. In addition, and as will be 
described more fully below, the spectrum includes pay-per-view 
digital channels 106, and further additional definable digital channels 
108 offering a variety of interactive services, in addition to the general 
purpose or special channels or services particular to the specific 
application of the present invention. 

As illustrated in Figure 3b, the digital program listing 
channel 100 provides data representing daily and weekly program 
listings and related information from the service provider 50 to the 
transceiver 54. As shown in Figure 3b, the listing channel 100 
includes a repetitive data stream having data representing today's 
listing 110 interleaved with weekly listings. For example, there is 
shown today's data 110, followed by Monday's data 112, and then once 
again followed by today's data 110. As seen in Figure 3b, Tuesday's 
data 114 is followed once again by today's data 110, and thereafter by 
Wednesday's data 116. Wednesday's data 116 is followed by today's 
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data 110, and thereafter by Thursday's data 118. Thursday's data 118 is 
followed once again by today's data 110, and thereafter by Friday's data 
119. Finally, following today's data 110, Monday's data 112 is once 
again transmitted along the data stream provided by the service 
provider 50. For programming purposes, the data preferably will 
include titles of programs, show times, special captions, length 
information, categories, and key words, as well as channel numbers 
provided from the service provider 50 over the T/T cable 52, and 
received by the transceiver 54. The data stream is an interleaved data 
stream which repeats on a weekly basis as shown in Figure 3b. It is 
presently contemplated that there would be at least 100 different 
channels of programs and/or services. A weekly program/service 
listing with information relating to 100 channels of 
programs /services could require over 2 megabytes of information. 
This data stream of programs /services listing information (illustrated 
in Figure 3b) is received by the transceiver's main CPU module 62, 
whereafter the sections that are most relevant to the users are stored 
in the system memory 65. By interleaving the current day's data with 
the weekly data, the daily data, the CPU module 62 can update the 
system memory 65 periodically and still provides quick access to the 
viewer, without having to store all the received program /service 
information, thereby reducing the amount of system memory 65 
required and the associated hardware cost. 



WO 95/01057 



PCT/US94/06941 



32 

Referring once again to Figure 3a, the back channels 102 
are used to engage in a variety of transactions, such as ordering 
products, home banking services and pay-per-view movies, as will be 
described more fully below. In operation, any request by a user to 
view a pay-per-view movie or order a product is transmitted to the 
service provider 50 over at least one back channel 102 through the 
T/T cable 52. It is contemplated that as channel capacity increases, so 
will the number of back channels, allowing for broader band two-way 
communications such as telephone conversations or interaction with 
on-line networks. 

Referring now to Figures 4a, 4b, and Figures 5a and 5b, 
two alternative embodiments of the remote control device 60 are 
shown in further detail. It is expected that the model shown in 
Figures 5a and 5b would be more commonly used. As illustrated in 
the figures, both embodiments of the remote control device 60 are 
comprised of four control button groups 115, 134, 160 and 176. These 
control button groups 115, 134, 160 and 176 will first be described 
briefly here, and their usage will be described in further detail below. 

The first control button group is the basic control button 
group 115 which includes standard television control buttons such as 
power on/ off button 120, volume control button 122 and a mute 
button 125. Moreover, channel up/down buttons 130 are provided to 
the embodiment illustrated in Figures 4a and 4b to increment or 
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decrement the currently viewed channel on television 58, as is well 
known. For the embodiment illustrated in Figures 5a and 5b, the 
channel control function is effectuated using the up and down arrow 
buttons 145 and 146. 

The second control button group is the interactive 
control button group 134 which includes an info button 136, a list 
button 138, a categories button 140, and a mark button 142. The info 
button 136 permits the user to obtain multiple levels of information 
on programs while the user is watching one of the program offerings 
on the TV 58 (or other A/V devices). The list button 138 permits the 
user to view listings for programs/services. In the presently preferred 
embodiment, when TV is selected, the list button 138 permits the user 
to view current and future TV listings up to one week in advance. 
The categories button 140 permits the viewer to select 
programs/services by category. For example, and as will be described 
below, the user may desire to watch only sports programs, only news 
programs, or the like. As will be disclosed, the categories button 140, 
and the list button 138 operate in concert to provide listings of 
selected categories, such as sports or news programs, on the TV 58. 
The mark button 142 permits the user to mark programs for 
reminders, later recall, or switching between programs which have 
been selected using the mark button 142. A jump button 132 lets the 
user switch to the program that was last marked, then to the program 
that was marked before that, and so on, until all marked programs 
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have been shown, looping back to the last marked program again. As 
shown in Figures 5a and 5b, the jump button 132 and the mark button 
142 are placed side by side together on the dividing line between the 
first and second control button groups 115 and 134 to emphasize their 
interrelation. 

Also provided with the interactive control button group 
134 is a pointing device consisting of an up arrow button 145, a down 
arrow button 146, a left arrow button 148, and a right arrow button 
150. For example, if the user is viewing a program /service listing 
through the use of list button 138, and desires to scroll up or down 
within the listing, the up arrow button 145 and the down arrow 
button 146 are utilized. In addition, a center select button 155 is 
provided for permitting the user to select one of several choices, 
represented by words or icons in graphic menus or lists displayed on 
the TV 58, for example a "bull's eye" as shown in Figure 5a. As will 
be described, the interactive control button group 134 may be used in a 
variety of applications, including selection of pay-per-view channels, 
home shopping services, and the like, as well as to display and control 
programs from other A/V devices connected to the transceiver 54. 

The third control button group is the auxiliary control 
button group 160 comprising various A/V control buttons for 
controlling the VCR 56 or the other A/V devices 57 connected to the 
transceiver 54. As illustrated in Figure 5a, the auxiliary control 
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button group 160 includes a record button 162, a rewind button 164, a 
stop arrow button 166, a play /pause button 168, a fast forward button 
170, and a menu button 172. In accordance with the teachings of the 
present invention, the user depresses the record button 162 while 
watching a television program on TV 58, which brings up a record 
panel confirming the current program's title and length, along with a 
highlighted select icon and confirmation of which A/V device and 
type to use for recording. Once the center select button 155 is 
depressed, the VCR 56 or the other selected A/V device 57 begins to 
record the program currently being viewed by the user. Alternatively, 
if the user has pressed list button 138, and a program listing is 
highlighted, pressing the record button 162, and then the center select 
button 155 results in the VCR 56 being programmed to record the 
particular show highlighted in the program listing. Pressing the 
menu button 172 displays icons for all the A/V devices 57 currently 
connected to the transceiver 54 and allows users to select another 
device with the arrow buttons. Pressing the center select button 155 
displays the output of that device and lets the user control that device. 

The fourth control button group is the numeric keypad 
panel 176, which permits the user to directly enter channel numbers 
to be viewed on the TV 58 by pressing a single, double or triple digit 
number, followed by the ok button 178 (Fig. 5a). For the remote 
control device 60 shown in in Fig 4b, the user would depress the select 
button 155 after entering values using the keypad 176. In addition, 
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the numeric keypad 176 may be used by the user to enter a personal 
identification number ("PIN"), to be transmitted to the service 
provider 50 through the A/V connect module 66 for payment of pay- 
per-view movies, products and the like. 

In addition to the control button groups, both 
embodiments comprise a transmitter (not shown) for transmitting 
signals to the audio-visual system. Additionally, the model 
illustrated in Figures 5a - 5b is further provided with a talk button 181, 
a microphone 179, a speaker 182 and a receiver (not shown). The talk 
button 181 when depressed allows the user to speak into the 
microphone 179, thereby providing input to the A/V system through 
voice, and receive voice feedback from the A/V system through the 
speaker 182. 
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Full Screen Viewing Function 

Referring now to Figure 6, there is shown a 
representative full screen audio-visual program displayed on the 
screen 180 of the TV 58. By depressing the channel up/down buttons 
130 on the remote in Figures 4a and 4b, (or by depressing the up and 
down arrow buttons 145 and 146 on the remote control device in 
Figures 5a - 5b), users can switch channels as is commonly done with 
regular TV remotes. As the user selects a channel to view using the 
remote control device 60, a graphic overlay panel 185 is displayed in 
the lower right hand portion of screen 180. As illustrated, the graphic 
overlay panel 185 includes the channel number currently being 
viewed 186 along with the channel logo and identifier 188. 
Additionally, a marking identifier 189, such as a check mark, is also 
displayed, as will be described, to indicate that the program has been 
marked through the use of the mark button 142 previously discussed 
with reference to the remote control device of Figures 4a, 4b, and 
Figures 5a and 5b. 
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Information Function 

Referring now to Figure 7, there is shown the screen 180 
in which the information (info) function 136 has been requested by 
the user. As the user is watching a particular television program on 
the TV 58, he may depress the info button 136 to obtain information 
in the form of a graphic overlay panel 190 on the screen 180. In 
operation, the transceiver 54 coupled with the main module 62 
through the A/V connect module 66 superimposes the graphic 
overlay panel 190 against the currently displayed video image. This 
process is also used in other functions, such as the full screen viewing 
function, where the graphic overlay panel 185 is superimposed on the 
screen 180. The overlay graphic panel 190 contains an "i" logo 192 
indicating that the information button 136 has been depressed, and 
also displays the title of the currently viewed program 194 ("Showbits 
This Month" in this example), a category name 196 ("News" in this 
example), and the length of the overall program ("60 min" in this 
example). In addition, the channel number and network logo ("8" 
and "CMM" in this example) are also provided to the user. Also, as 
illustrated in Figure 7, the overall length of the currently viewed 
program is graphically illustrated, by a time bar 200 showing how 
much time remains before the program ends. A left arrow 201, a right 
arrow 202, and a down arrow 210 are displayed and highlighted to 
indicate that the left arrow button 148, the right arrow button 150, and 
the down arrow button 146, of the remote control device 60 can be 
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depressed to cause additional information to be depressed. An up 
arrow 205 is also displayed, but is not highlighted, to indicate that no 
additional information will be displayed if the up arrow button 145 is 
depressed. 

Referring now to Figure 8, which illustrates a new 
graphic overlay panel 190 displayed in response to the user having 
depressed the right arrow button 150 on the remote control device 60 
while using the information function. The transceiver 54 provides 
additional information on the currently viewed program. As 
illustrated, an additional page of information on the program 
currently viewed is displayed. For example, additional guests on the 
show "Showbits This Month" include "Madeline, Harold Black and 
Harmer". Referring now to Figure 9, which illustrates another new 
graphic overlay panel 190 displayed in response to the user having 
depressed the right arrow button 150 again. A third page of 
information is displayed on the screen 180 relating to the currently 
viewed program "Showbits This Month". It will be noted that the 
currently viewed television program ("Showbits This Month" in this 
example) continues to be viewed in the main portion of the screen 
180, while different graphic overlay panels 190 are superimposed over 
the lower third portion of the screen 180. 

In the currently preferred embodiment, a user may 
selectively move between pages of information relating to the 
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currently viewed program by pressing the right arrow button 150 or 
the left arrow button 148 on the remote control device 60, or 
alternatively, the main module 62 will automatically scroll through 
the various pages of information with a pre-determined amount of 
display time for each page of information. 

Referring now to Figure 10, there is shown a currently 
viewed baseball game entitled "Saint Louis at San Francisco" 
displayed on the screen 180. Assume now the user also depresses the 
info button 136 on the remote control device 60. As previously 
described, information relating to the currently viewed program is 
displayed within the graphic overlay panel 190. (n the present 
example, information relating to the baseball player, "John Smith", is 
provided within the overlay 190. In the event the down arrow 210 is 
highlighted, an additional information track is available to the user. 
This additional information track may be displayed through the use 
of down arrow button 146 on the remote control device 60, and is 
referred to here as "Tid Bits". As shown in Figure 10, the "Tid Bits" 
information track gives additional information relating to the 
particular subject currently being displayed on the screen 180. In the 
example of Figure 10, the first baseman John Smith is currently being 
viewed on the screen 180, and information relating to John Smith is 
provided in the graphic overlay panel 190. It will also be noted that in 
Figure 10, the down arrow 210 is highlighted, indicating that yet 
another information track is available. 
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Still referring to Figure 10, it will be noted that a down 
arrow 210 is shown. By depressing the down arrow button 146 on the 
remote control device 60 once again, an Info Mart track information 
track as shown in Figure 11 is displayed by CPU 63 in the graphic 
overlay panel 190. As illustrated, the Info Mart information track 
provides the viewer with the opportunity to purchase products, 
services, programs and the like related to the program which is 
currently being viewed (in the present example, the baseball game 
"Saint Louis at San Francisco"). The information track disappears 
when the info button 136 is pressed again. 

While the information has been described with the left 
and arrow arrow buttons 148 and 150 causing different pages of an 
information track to be displayed, and the up and down arrow 
buttons 145 and 146 causing different information tracks to be 
displayed, it is contemplated that the information displayed may be 
related to each other in other application dependent manners. It is 
further contemplated that this information function can also be used 
with audio-visual programs or services other than TV shows, in 
particular when the menu button 172 is used to display the output of 
the VCR 56, the CD 70, a hard disk, a telephone or another other A/V 
device 57 connected to the transceiver 54. 
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The data comprising the information of the various 
information tracks shown in Figures 7 - 11 are downloaded using the 
method illustrated in Figure 3b, i.e. the interleaved data stream 
transmitted from the service provider 50, to the transceiver 54. The 
data are stored in the system memory 65 and updated as new data are 
received over the digital program listing channel 100. Alternatively, 
data displayed in the information tracks such as the exemplary Tid 
Bits or Info-Mart information tracks may be transmitted by the service 
provider 50 to the transceiver 54 using the vertical blanking intervals, 
which are inherently part of the analog TV channel 103 used to 
display the currently viewed program on the TV 58. In another 
embodiment, data comprising the information tracks may be 
transmitted and downloaded through the use of separate info-track 
channels comprising additional channels in the electronic spectrum 
illustrated in Figure 3a. 
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List Function 

Referring now to Figure 12, the list function of the 
present invention will be described. While viewing a television 
program displayed on the TV 58, the user may depress the list button 
138 on the remote control device 60 to obtain a program/ service 
listing for the current date and time during which the user is 
watching television. As shown in Figure 12, upon depressing the list 
button 138, a program/ service listing 220 is displayed for the current 
date and time at which the viewer has depressed the list button 138. 
As illustrated, programs /services currently available for viewing and 
interaction are identified by their channel numbers, channel name 
identifiers (for example, Channel 10, NTV), and by titles of the 
programs /services. In the event that the program /service listing 
includes two programs for a given time interval, then the names of 
both programs are displayed. For example, in Figure 12, Channel 10 is 
NTV which beginning at 8:00 p.m. is showing a program entitled 
"Rock Today", and beginning at 8:10 p.m. is showing a program called 
"Party Time". In addition, as shown in Figure 12, a picture-in-picture 
window 250 continues to display the currently viewed program 
which the user was last viewing. Accordingly, the user may continue 
to view the currently selected program, and the current 
program /service listing simultaneously. By pressing the select button 
155 in the current time slot, the user can view the currently 
highlighted program in full screen, as long as that program is playing 
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live at the current time. For example, in the case of Figure 13, the 
pressing of the select button 155 would display NTV's "Rock Today" 
program in full screen. It will be noted that although the current 
implementation of the list function displays a list of 
programs /services on single lines for one hour time slots, the 
invention's user interface also supports other styles of display, for 
example, a grid showing programs on two lines, each line 
representing a two hour time slot. 

It will also be noted that the currently selected program 
being displayed in the picture-in-picture window 250 is initially 
highlighted in the program/service listing to assist the user. In the 
example of Figure 12, Channel 11, PBS, a show entitled "Say Nay 
Billy" is highlighted. Moreover, channels may be selected by pressing 
the up arrow button 145 or the down arrow button 146 on the remote 
control device 60 to move the highlighting over to alternate 
program/ service offerings. For example, if the user presses the down 
arrow button 146, as shown in Figure 13, the NTV listing is then 
highlighted. Similarly, as shown in Figure 14, by continuing to 
depress the down arrow button 146, the highlighting continues to 
move upward towards the lower channel numbers and once the 
highlighting reaches Channel 9, the entire program /service listing 
scrolls to display consecutively lower channel numbers previously 
not displayed. At the same time, the higher channel numbers 
previously displayed at the bottom part of the screen 180 will 
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consecutively disappear. As illustrated in Figure 14, Channels 7 and 8 
are now displayed, and Channels 19 and 20 are no longer displayed to 
the user. If the up arrow button 145 is now pressed instead, the 
highlighting will move down towards the higher channel numbers 
as illustrated in Figure 14. Notably, movement of the 
program/service listing and the highlighting in the present 
invention are specifically calculated to conform with an average 
person's expectations in using a television remote control. While the 
list function has been described with the down arrow button 146 
causing highlighting to move towards the lower channel numbers, 
and the up arrow button 145 causing highlighting to move towards 
the higher channel numbers, it will be appreciated that the 
correspondence between the arrow buttons 145 and 146 and the 
highlighting movement may be implemented in other ways as best 
suits the anticipated user of the interface system. 

Skipping now to Figure 16, a further feature of the list 
function of the present invention is illustrated. In the event the user 
depresses the right arrow button 150, the CPU 63 displays the next 
hourly time slot listing on the screen 180. In the example of Figure 
16, the time slot for 9:00 - 10:00 p.m. is illustrated. It will be 
appreciated that, had the user depressed the left arrow button 148 
three times, then the program/service listing for 6:00 - 7:00 p.m. 
would have been displayed. Additionally, as illustrated in Figure 17, 
the continued depression of the right arrow button 150 for a 
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predetermined amount of time (in the present embodiment for more 
than 2 seconds) results in the display of program /service listings for 
subsequent days. In this case, the continued depression of the right 
arrow button 150 has resulted in a scanning of the program /service 
listings through to Saturday, October 17. It will be noted that in the 
currently preferred embodiment, a scanning of subsequent days 
results in the display of the program /service listing from the 
beginning of prime time, namely, 6:00 p.m. (or 7:00 p.m., at the 
service provider's option). It will also be noted that the info function 
can be used in conjunction with the list function to display 
program /service information about the currently highlighted 
program/ service. It will further be noted that the listing display will 
disappear if the user depresses the list button 138 once again. It is also 
contemplated that this list function can be used with audio-visual 
programs or services other than TV programs, in particular when the 
menu button 172 is used to display the output of other A/V devices 
57. For example, if the VCR 56 is displayed on the TV 58, pressing the 
list button 138 on the remote control device 60 would display a listing 
of all programs recorded by the user on the VCR 56, highlighting the 
program now displayed from the VCR 56. Similarly, the list function 
would display CD ROM or telephone listings when these A/V devices 
are selected through the menu function. 
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Record Function 

Referring now to Figure 18, which illustrates a 
program /service listing displayed in response to the user having 
continuously depressed the right arrow button 150 on the remote 
control device 60. The exemplary program/ service listing displayed is 
for Saturday, October 17 beginning at prime time (6:00 p.m.). The up 
and down arrow buttons 145 and 146 on the remote control 60 are 
then used to highlight the program "Specials Tonight". Assume now 
the user desires to record the exemplary program "Specials Tonight". 
By pressing the record button 162 on the remote control device 60, a 
record panel 259 is displayed by the CPU 63 on the screen 180, 
confirming the title and length of the program to be recorded, along 
with a highlighted select ("bull's eye") icon as shown in Figure 18. It 
is contemplated that other functions can be included in the record 
panel 259, such as a function for selecting different VCRs or other 
A/V recording devices, a function for selecting different tapes or 
reading media, and/or a function for selecting how often the 
recording is to take place (once, weekly, daily). Once the select button 
155 is pressed, the program listing is redisplayed without the record 
panel 259. A record icon 260 is displayed on the screen 180, as shown 
in Figure 19. The "Specials Tonight" program which will begin on 
Saturday, October 17, will then be recorded automatically. In the 
event the viewer changes his mind and does not desire to record the 
program "Specials Tonight", by pressing the record button 162 again 
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and then pressing the select button 155 once more, the program 
listing will be redisplayed without the record icon 260 and no 
recording will take place. 
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Mark and |ump Functions 

Skipping now to Figure 15, the mark function of the 
present invention and its operation will be described. Shown in 
Figure 15 is a program listing reflecting Channel 10 (NTV) having 
been selected using the up and down arrow buttons 145 and 146 on 
the remote control device 60. Assume now the current time is 8:08:30 
p.m. In accordance with the teachings of the present invention as 
described earlier, the program "Rock Today" would be highlighted on 
the screen 180. The program "Party Time" may be highlighted by the 
viewer by pressing the right arrow button 150. The mark function 
may then be initiated by pressing the mark button 142 on the remote 
control device 160. As illustrated in Figure 15, a mark identifier 252 is 
displayed adjacent to the selected program (in the present example, 
"Party Time"). Skipping again to Figure 20, a reminder box 254 is 
displayed on the screen 180 at the time the program "Party Time" 
begins (in the present example, 8:10 p.m.), regardless of what channel 
is currently being viewed then by the user on TV 58. It should also be 
noted that programs playing live in a current time slot can be marked 
while displayed in full screen by depressing the mark button 142. 
This causes a mark identifier 189 to be displayed in the graphic 
overlay panel 185, as shown in Figures 6 and 21. 

Referring now to Figures 20 and 21, depressing the jump 
button 132 while the reminder PIP window 254 is displayed switches 
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the program displayed in the main viewing area of the screen 180 to 
the program which the user has previously marked (see Figure 21). If 
more than one program has been marked, by repetitively depressing 
the jump button 132, users can jump to the other previously marked 
programs displaying them in full screen in the main display area of 
the screen 180 one at a time. It will be appreciated that in an alternate 
embodiment, the picture-in-picture window 254 illustrated in Figure 
20 may be preceded by a reminder icon displayed automatically in the 
PIP window 254 a few minutes prior to the actual start of the program. 
It will also be noted that the mark and jump functions may be used 
with audio-visual programs other than TV shows, allowing a user to 
mark and subsequently jump between, for example, a VCR program, 
a CD ROM program, an on-line service or an answering machine 
message, in addition to the TV programs previously marked while 
viewing television channels. 
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Categories Function 

Referring now to Figures 22 - 32, the categories function 
of the present invention will be described. To initiate the categories 
function of the present invention, the categories button 140 is 
depressed on the remote control device 60. As illustrated in Figure 
22, the CPU 63. displays a categories graphic overlay panel 300 which is 
superimposed over the currently viewed program on screen 180. A 
plurality of categories are identified by representative icons. The first 
category which is selected by default is the "all" category 302, and is 
highlighted around the border. The "all" category 302 refers to all 
currently available programs/services being transmitted by the 
service provider 50. By depressing the right arrow button 150 or left 
arrow button 148 on the remote control device 60, other categories 
may be highlighted. For example, by depressing the right arrow 
button 150, the next category "favorites", as indicated by a star icon 
305, is highlighted (See Figure 23). Similarly, by depressing the right 
arrow button 150 again, a movies icon 310 is highlighted. By 
depressing the right arrow button 150 another time, a sports icon 315 
is highlighted (see Figure 24), and so on. Additional categories may 
include a news icon 312 and a music video icon 313. However, it will 
be appreciated that many other categories can be displayed. By 
repeatedly pressing the right arrow button 150, the user can make 
category icons appear, scrolling from right to left against the category 
overlay panel 300. 
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In Figure 24, the sports category has been highlighted by 
the user as indicated by the highlighted sports icon 315. To select a 
category that is currently highlighted, the user can press the select 
button 155 and the category overlay panel 300 disappears while the 
currently viewed program displayed on the screen 180 is replaced by 
the nearest available program in the category just selected. The 
selection of the sports category results in the display of only sports 
related programs on the TV 58. Figure 25 illustrates an exemplary 
baseball game which is displayed after the user presses the select 
burton 155. As illustrated, the sports icon 330 is also displayed in the 
graphic overlay panel 185 along with the channel identifier 188. 

Assume now after the nearest sports program has been 
displayed on the screen 180, the user desires to view additional sports 
category programs on the TV 58. By depressing the up channel button 
130 or the up arrow button 145 on the remote control device 60, the 
transceiver 54 displays the next higher number channel which is 
currently showing a sports program. In the example of Figure 26, the 
sports program relates to a windsurfing event. As shown, the sports 
icon 330 continues to be displayed along with the numerical channel 
indicator and the channel identifier 188 (in the example of Figure 26 
"Sportstime") in the graphic overlay panel 185. 
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Referring now to Figure 27, if the user has selected the 
category of sports (icon 315), and depresses the list button 138, a 
program/service listing 360 of currently available sports programs is 
displayed on screen 180. It will also be noted that the currently 
viewed sports program will continue to be displayed in a picture-in- 
picture window 365. As previously discussed, with respect to the list 
function, programs which are shown during half hour intervals (See 
Figure 27 "Windsurfing" and "Basketball") are identified and may be 
individually selected using the remote control device 60. 

Referring now to Figure 28, which illustrates a program 
listing with the sports icon 330 in the category overlay panel 
highlighted, displayed in response to the user having pressed the list 
button 138, and then presses the categories button 140. Assume now 
the user desires to break down the main sports category into finer sub- 
categories such as baseball, football, basketball, hockey, etc. By 
depressing the down arrow button 146 on the remote control device 
60, the user causes the CPU 63 to select a default sport sub-category 
(baseball in this example) and display a plurality of sports sub-category 
icons against the categories overlay panel 300 as illustrated in Figure 
29. The default sports sub-category icon 315 is also highlighted, and 
the program listing 361 comprises only programs of that sports sub- 
category, i.e. baseball in this example. By pressing the right arrow 
button 150 on the remote control device 60, the user can highlight the 
football sub-category icon 317, then press the select button 155 to cause 
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only football programs to be listed. Alternatively, the user can press 
the up arrow button 145 to display the main categories again, with the 
sports icon 315 highlighted as illustrated in Figure 28. The user can 
also press the categories button 140 once more to make the categories 
overlay panel 300 disappear. 

Referring now to Figure 30, which illustrates a program 
listing 320 with the favorite icon 305 in the category overlay panel 
highlighted, displayed in response to the use having selected the 
"favorites" category using the left arrow button 148 on the remote 
control device 60. The program listing 320 comprises favorite 
programs for the current time slot including frequently viewed 
programs, marked programs, and programs to be recorded, as well as 
programs that match specified user preferences. Assume now that 
the user desires to resolve the "favorites" category into finer sub- 
categories. By pressing the down arrow button 146 on the remote 
control device 60, the user causes the CPU 63 to display a plurality of 
favorites sub-category icons against the categories overlay panel 300, 
as illustrated in Figure 31. The CPU 63 highlights the "frequent" 
programs sub-category icon 318 and displays a program listing 362 of 
programs most frequently watched by the user as determined by the 
CPU 63 from previous monitoring of the user's viewing habits. In 
the normal course of operation, the CPU 63 monitors programs 
which are most frequently viewed and compiles a prioritized list of 
those frequent programs for each time slot, which list is stored in the 
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rewritable non-volatile part of the system random access memory 
(such as SRAM) 65. 

In the presently preferred embodiment, if a viewer 
views a television program for more than 10 minutes, the current 
program and channel identifiers are compared to frequent programs 
already stored in system memory 65 for that particular time slot, and 
an updated list of frequent programs is created by CPU 63 and stored 
in the system memory 65. In the event there are no frequent 
programs stored in the system memory 65 for any particular time slot, 
then a general list of most frequently watched programs is retrieved 
from the system memory 65. It is anticipated that the majority of 
frequent programs will be stored in time slots corresponding to prime 
time, or during daytime television hours. 

By pressing the right arrow button 150, the user can 
move the highlighting from the "frequent" programs sub-category 
icon 318 to the "marked programs" sub-category icon 319, as shown 
on Figure 32. Accordingly, the CPU 63 highlights the "marked 
programs" icon 319 and displays a program listing 363 of only 
programs that have been previously marked. Other favorite sub- 
categories include a record sub-category of all programs selected for 
recording, and personalized favorites sub-categories such as "Mom" 
321 and "Joe" 322. It is contemplated that new preferences can be 
defined by individual users by selecting a special favorites sub- 
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category, represented by a "set-up" icon 323, then marking preferred 
items from a list of programs and/or categories, as well as specifying 
an icon and/ or a name for this new preference set. By first defining 
and subsequently selecting, for example, the "Mom" sub-category icon 
321, a user can view only programs that match the interests defined 
earlier by the mother of the household. 

It is also contemplated that this category function can be 
used in combination with A/V programs or services other than TV 
programs, in particular when the menu button 172 is used to select 
the VCR 56, or one of the other A/V devices 57. For example, 
pressing the categories button 140 while viewing a VCR listing (Fig. 
36, below) would enable users to find all previously recorded comedy 
programs. Similarly, the categories function could enable users to 
find phone listings in the "gardening" category, according to 
telephone directory information accessed from the provider 50 via 
the T/T cable 52. 

While the present invention has been described with the 
categories function having different effects on the displayed image 
when used during full screen display and when used in conjunction 
with list function, it will be appreciated that the present invention 
also may be practiced with the categories function having consistent 
effect on the displayed image in either situation, as desired. 
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Pix Function 

Referring now to Figures 33 and 34, the present 
invention's picture-in-picture (or pix function) will now be described. 
The depression of the pix button 144 on the remote control device 60 
results in the display of a pix display 381 as illustrated in Figure 33. As 
shown in the present embodiment, 12 picture-in-picture video 
windows are arranged within the screen 180, along the perimeter of 
the pix display 381, with a central picture-in-picture window 375 also 
displayed. Depressing the pix button 144 on the remote control device 
60 results in the simultaneous display of multiple programs currently 
transmitted over the T/T cable 52, the VCR 56, the CD ROM 70 and/or 
the other A/V devices 57. Moreover, one of the picture-in-picture 
windows displaying the last program viewed in full screen, such as, 
for example, picture-in-picture window 380, is highlighted. As shown 
in Figure 33, the highlighted smaller screen is also displayed in larger 
size in the central picture-in-picture window 375, with the audio track 
of the program also audible to the user. Accordingly, a user may 
watch a motion picture or television program, including the audio 
track, in window 375, while viewing multiple other programs 
simultaneously. In the presently preferred embodiment, the A/V 
connect module 66 samples (or "grabs") a video frame from each of 
the 12 programs every pre-determined (N) number of frames. 
Accordingly, a user viewing the pix display 381 perceives each of the 
12 small picture-in-picture windows as sequential representative 
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images of every N frames per window. However, the picture-in- 
picture window 375 represents a full motion display of the currently 
selected program, provided that a second TV tuner 67 is included 
within the A/V connect module 66. 

Additionally, in the preferred embodiment of the 
present invention, it is contemplated that the user is able to view the 
pix display 381 by category by pressing the categories button 140 and 
selecting a particular category, such as favorites or movies. For 
example, the pix display 381 illustrated in Figure 33 is comprised of 
only favorite programs, as the user had previously selected the 
favorites category. Had the user selected the "marked" sub-category, 
only marked programs would be displayed on the pix display 381, 
allowing users to easily add or delete programs in the pix display 381 
by using the mark button 142. If no categories have been selected 
(which is equivalent to selecting the "all programs" category), then 
the pix display 381 displays the 12 programs nearest to the last selected 
program. Similarly, pressing the info button 136 would display the 
graphic overlay panel 190 shown in Figure 7 for program 
information. 

Referring now to Figure 34, which illustrates the 
selection of a different pix program to be viewed in the central 
picture-in-picture window 375. By depressing the left arrow button 
148 on the remote control device 60, the highlighting 380 is moved 
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one window to the left. In the example of Figure 34, a "Discover It" 
program is now highlighted and shown simultaneously in the central 
picture-in-picture window 375. Similarly, through the use of the left 
arrow button 148 and the right arrow button 150 along with the up 
arrow button 145 and the down arrow button 146 on the remote 
control device 60, each of the pix programs in the pix display 381 may 
be viewed within the central picture-in-picture window 375, as 
desired. As in the list function, pressing the select button 155 displays 
the currently highlighted program in full screen. Alternatively, 
pressing the pix button 144 a second time causes the pix display 381 to 
disappear and the last highlighted program appears in full screen. 

Referring to Figure 35, selection of pay-per-view movie 
currently shown in one of the picture-in-picture window 380 of the 
pix display 381 for viewing is illustrated. As described earlier, by 
depressing the left arrow button 148 on the remote control device 60, 
the PIP window 380 becomes highlighted. In the example of Figure 
32, the pay-per-view channel shows a preview trailer for the motion 
picture "Jeff's World", which is now displayed in the center PIP 
window 375 and the perimeter PIP window 380 simultaneously. To 
select the pay-per-view movie, the select button 155 is depressed on 
the remote control device 60. It is contemplated that pressing a single, 
double or triple digit number on the numeric keypad 176 , followed by 
the ok button 178 of the remote control 60, will display the 
corresponding pay-per-view channel, along with the 12 nearest 
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channels. It is also contemplated that this pix function can be used 
with audio-visual programs and services other than TV shows, in 
particular when the menu button 172 is used to display the output of 
the VCR 56, the CD 70, a hard disk or one of the other A/V devices 57 
connected to the A/V transceiver 54. 
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Menu Functign 

Referring now to Figure 36, the menu function will now 
be described. The menu function permits users to switch the 
television display to any A/V source connected to the transceiver 54, 
such as the VCR 56, the CD ROM module 70, or one of the other A/V 
devices 57, and lets users control those A/V sources directly. As 
shown in Figure 36, upon pressing the menu button 172 on the 
remote control device 60, the CPU 63 displays on the screen 180 a 
graphic overlay panel 420 which is superimposed over the currently 
viewed audio-visual program. As further shown in Figure 36, a 
plurality of audio-visual sources connected to the transceiver 54 are 
identified by representative icons. The currently viewed A/V source 
icon is automatically highlighted, typically the TV icon 421. By 
depressing the right arrow button 150, or left arrow button 148 on the 
remote control 60, other A/V source icons may be highlighted and 
previewed. For example, by depressing the right arrow button 150, 
the next A/V source icon, VCR 422, is highlighted, as shown in 
Figure 36. Simultaneously, CPU 63 displays an audio-visual program 
derived from the highlighted source icon, in this case a videotape 
playing in the VCR 56 onto the screen 180, as illustrated in Figure 36. 
To select and control any A/V source whose icon is currently 
highlighted, in this case the VCR 56, the user can press the select 
button 155, whereafter the graphic overlay panel 420 disappears while 
the currently viewed program sourced from the VCR 56 is displayed 
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in full screen. Thereafter, the user can press the play button 168 on 
the remote control device 60 to cause the CPU 63 to instruct the VCR 
56 through the A/V connect module 66 or the IR control 82 to 
playback a previously recorded program on the VCR 56. Similarly, 
the user can press the stop arrow button 166, the rewind button 164, 
and/or the fast-forward button 170 on the remote control device 60 to 
further control the program sourced from VCR 56. It is contemplated 
that the CD ROM module 70 and some of the other A/V devices 57 
including other CDs, VCRs, laser disc players, hard disks, telephones, 
answering machines, as well as transmitting devices connected to 
voice mail, electronic mail and various on-line services can be 
similarly selected and controlled through the use of the menu 
function, as illustrated in Figure 36. 
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Other Functions 

Other functions may be realized with the present 
invention, including numeric data entry. As shown in Figures 41 
and 49, by pressing one or more buttons on the numeric keypad 176 of 
the remote control device 60, a user can confirm a financial 
transaction by entering his or her personal identification number 
(PIN) in a confirmation display field 425. The numeric keypad 176 
may also be used for dialing TV channel numbers, telephone 
numbers and other program source identifiers, as desired. 

Yet another function which may be optionally realized 
by the present invention is a talk function, whereby a user can depress 
the talk button 181 on the remote control device 60, and thereafter 
issue spoken commands into the microphone 179 located on the 
remote control device 60 in lieu of or in addition to pressing buttons 
on the remote control device 60. Through the use of voice 
recognition hardware and software operating in conjunction with the 
CPU 63, the present invention can be made to interpret the spoken 
commands requested by the user, and invoke the corresponding 
functions. For example, a user may speak the word "sports" while 
pressing talk button 181, which is equivalent to invoking the category 
function and highlighting the sports category, as if the user had 
pressed the corresponding sequence of buttons, as described earlier. It 
is further contemplated that the remote control device 60 in 
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conjunction with the TV 58 and a camera can be used as a video 
telephone for video conferencing and the like, wherein having 
engaged a telephone line (via the menu function) the user speaks 
directly into the remote control device 60 as if it were a standard 
telephone. 
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Pay-Per-View Infcrf ace 

The present invention includes one possible pay-per- 
view (PPV) interface which will now be described. Assume that the 
user has selected the TV source icon 421 using the menu function 
displayed in the graphic overlay panel 420, and further selects a pay- 
per-view program using either the list or the pix function. This 
selection results in an initial title screen to be displayed, as illustrated 
in Figure 37, which may be static or animated. As shown in Figure 38, 
upon the selection of the PPV program and subsequent to the display 
of the title screen of Figure 37, a preview icon 382, a credit icon 384, an 
info icon 386, and a ticket icon 388 are displayed, along with a preview 
trailer which is continuously looping and is displayed in a half-screen 
picture-in-picture window 390. It will be noted that in Figure 38, the 
preview icon 382 has been highlighted by default, thereby 
corresponding to the trailer being shown in the window 390. 

Referring now to Figure 39, by depressing the right arrow 
button 150 of the remote control device 60, the credit icon 384 may be 
highlighted. The credits for the pay-per-view movie are displayed on 
the screen 180 within the window 390. By depressing the right arrow 
button 150 once again, the info icon 386 is highlighted. As shown in 
Figure 40, information related to the pay-per-view movie is then 
provided to the user, such as the title, the production company, a brief 
description of the movie, the rating and the total length. If the user 
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desires to watch the pay-per-view movie, he/ she depresses the right 
arrow button 150 once again to highlight the ticket icon 388, as 
illustrated in Figure 41. 

As shown in Figure 41, upon the selection of the 
highlighted "ticket" icon 388, the CPU 63 generates and displays on 
the screen 180 a request for the user to input a "movie pass" password 
using the numeric keypad 176 of the remote control device 60. After 
entering the personal identification number (PIN) using the numeric 
keypad 176, causing a confirmation sound to be played and 
confirmation symbols such as asterisks to fill in a blank confirmation 
display 425, the user depresses the ok button 178 or the select button 
155 on the remote control device 60 to order and view the pay-per- 
view movie. As shown in Figure 42, transceiver 54 then displays the 
ordered program if it is available, or displays a message informing the 
viewer that the pay-per-view movie will begin screening within X 
number of minutes. In operation, depressing the ok button 178 or the 
select button 155 subsequent to the entering of the movie pass 
personal identification number (PIN) results in the CPU 63 providing 
a signal to the cable service provider 50, over the T/T cable 52 using 
one of the back channels 102 (See Figure 3a). It will further be 
appreciated that during the waiting period prior to the transmission 
of the pay-per-view movie, additional trailers and other previews 
may be displayed to the user on the screen 180. It will also be noted 
that if the user incorrectly enters his/her movie pass PIN number, 
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pressing the clear button 177 on the numeric keypad 176 allows them 
to clear the confirmation display 425 and start over again. It will also 
be appreciated that other pay-per-view interfaces may be designed and 
implemented using the teachings of the present invention on general 
purpose audio-visual user interface. 
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Home Shopping Intgrf ?cg 

The present invention includes one possible home 
shopping interface, which will now be described. In Figure 43, a 
channel identified as "SHP" or "TV Shop" represents a dedicated 
home shopping service within the electronic spectrum illustrated in 
Figure 3a. It will be noted that although in its present embodiment, 
TV Shop is presented as a television channel, it could also be 
presented as an on-line service through a transmitting device 
connected to the T/T cable 52. In the current embodiment, the 
selection of the TV Shop service results in a continuous full-motion 
video display of various paid-for commercials or advertising 
messages. As shown in the representative screen of Figure 44, a TV 
Shop icon is displayed along with a blinking select icon (or right 
arrow icon) in a graphic overlay panel 400, at the start and end of each 
advertisement, prompting users to press the select button 155 (or right 
arrow 150). 

Referring now to Figure 45, by pressing the select button 
155 (or the right arrow button 150), the user can view additional 
information relating to the advertisements shown in the full motion 
video section of the TV Shop. In the example of Figure 45, it will be 
noted that a listing of the advertisers is provided, in which the 
advertisers whose commercial was playing last is highlighted. 
Through the use of the up arrow button 145 or the down arrow 
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button 146, the user may scroll through the advertisers, sequentially 
highlighting each of their advertising identifiers, along with a select 
icon 403 prompting a user to select that advertiser. Moreover, as 
shown in Figure 45, an "exit" icon 402, a "stores" icon 404, a "specials" 
icon 406, an info icon (or "compare" icon) 408, an "order" icon 409 
and other icons may be presented to the viewer. As illustrated in 
Figure 46 the selection of the "Attic at Casey's" advertiser with the 
select button 155 (or the right arrow button 150) results in a display of 
the various specials available from that advertiser. In Figure 46, there 
is shown cookware highlighted in conjunction with a still image of 
the cookware for sale. By depressing the down arrow button 146 on 
remote control device 60, the highlighted section on the display can 
be moved down such that the mini-espresso (see Figure 47) is 
selected. Simultaneously with the highlighting of the mini-espresso 
selection, a still image of the mini-espresso machine for sale is 
shown. 

Referring to Figure 48, to obtain further information 
regarding the mini-espresso machine illustrated in Figure 47, the user 
depresses the select button 155 (or the right arrow button 150) on the 
remote control device 60 to highlight the info icon 408. As illustrated 
in Figure 48, information relating to the particular product (i.e. 
espresso machine) is shown on the screen 180. To order the mini- 
espresso machine illustrated in Figures 47 and 48, the user depresses 
the select button 155 (or the right arrow button 150) to highlight the 
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order icon 409. As illustrated in Figure 49, a personal identification 
order number window 420 is displayed in which the user (using the 
numeric keypad 176) inputs a personal identification number (PEST). 
After entering the personal identification number (PIN), the ok 
button 178 or select button 155 is depressed to order the product. In 
the presently preferred embodiment, once the personal identification 
number (PIN) of the user is entered into the system of the present 
invention and the ok button 178 or the select button 155 is depressed, 
a signal is provided by the CPU 63 to the service provider 50, using 
one of the back channels 102 (See Figure 3)a. As illustrated in Figure 
50, a confirmation of the order, along with a delivery time is 
displayed to the user. It will also be appreciated that other home 
shopping interface variants may be designed and implemented using 
the teachings of the present invention. 
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Summary 

The present invention as described provides methods 
and apparatus for a unique audio-visual interface for television and 
other A/V programs and services. The present invention's functions 
and features provide a user interface which helps users find and view 
programs of interest, access related information, control various 
audio-visual devices and order products or services from their own 
homes. While the present invention has been described with 
references to the various figures , it will be appreciated that the figures 
are for illustration only, and do not limit the spirit and scope of the 
invention. For example, although the figures have, by necessity, used 
example screens having certain attributes, icons, criteria or functions, 
it will be appreciated that the invention is not limited by the specific 
examples provided. Rather, the present invention has application in 
any television and/or audio-visual display system. In addition, it will 
be noted that the present invention may be realized using a variety of 
computer hardware and computer software, and is not limited to any 
particular hardware or software systems. 

Moreover, it will be appreciated that many of the 
functions and features of the present invention, such as graphic 
overlay panels, icons, and selection methods, are not static events and 
are perceived by the user as having motion, color, sound and/ or 
animation. Specifically, it will be appreciated that the present audio- 
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visual user interface also includes an audio interface using a plurality 
of sound icons and backgrounds, which compliment the visual 
interface and are played almost every time a function of the present 
invention is used, and which cannot be illustrated graphically here, 
but are part of the present invention. Due to the limitations of a 
written specification, the reader is referred to a videotape entitled 
"EZTV" submitted by the Applicants concurrent with the filing of the 
application on which this patent is based. 
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WE CLAIM: 

1. In an audio-visual (A/V) system comprising a visual display 
and an audio reproduction system, said A/V system further 
comprising a transceiver coupled to a plurality of A/V program and 
service offering sources and said display, and a control device coupled 
to said transceiver, a method for managing viewing of a plurality A/V 
program offerings and managing interaction with a plurality of 
service offerings provided by said A/V program and service offering 
sources, said method comprising the steps of: 

(a) receiving a plurality source control inputs and in response 
selecting at least one of said A/V program/ service offering sources; 

(b) receiving and storing offering information regarding A/V 
programs and services, if any, offered by said selected at least one A/V 
program and service offering source; 

(c) displaying said received offering information on said visual 
display; and 

(d) receiving a plurality of selection control inputs and in 
response selecting and displaying one of said A/V program/ service 
offerings of said selected at least one program and service offering 
source on said visual display. 
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